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Executive  Summary 

In  this  study  two  models  used  by  the  Programs,  Analysis  and  Evaluation  Directorate  (PAED), 
HQDA,  to  evaluate  program  trade-offs  among  research,  development,  and  acquisition  (RDA) 
programs  competing  for  limited  Army  resources  are  briefly  described,  compared  and  their 
likelihood  to  produce  similar  results  is  analyzed.  One  model,  the  Research,  Development,  and 
Acquisition  Alternatives  Analyzer  (RDA3)  was  created  by  TRADOC  Analysis  Command  - 
Operations  Analysis  Center  (TRAC-OAC).  The  other  model,  Value  Added  Analysis  (VAA),  was 
created  by  Concepts  Analysis  Agency  -  Force  Systems  Directorate.  These  two  models  approach 
the  RDA  program  trade-off  problem  in  somewhat  different  ways,  but  both  can  be  used  to  provide 
an  optimized  list  of  RDA  programs  which  should  be  included  in  the  Army  Program  Objective 
Memorandum  (POM)  for  a  given  Total  Obligation  Authority  (TOA). 

Dr.  Ballaschi,  Deputy  Director  of  PAED,  requested  that  the  Operations  Research  Center  (ORCEN) 
compare  the  two  models  to  determine  if  they  would  produce  contradictory  results.  His  concern 
was  that  if  both  models  produce  an  optimized  list  of  RDA  programs  for  inclusion  in  the  POM,  then 
there  should  not  be  significant  disagreement  between  the  output  of  the  two  models.  This  study  is 
based  on  written  descriptions  of  both  models  and  conversations  with  personnel  knowledgeable  of 
each  model.  The  ORCEN  did  not  run  each  model  on  a  similar  set  of  inputs  and  subsequently 
evaluate  the  output. 

RDA3  and  VAA  were  found  to  have  taken  substantially  different  approaches  to  addressing  the 
problem  of  recommending  an  optimum  acquisition  strategy.  Given  these  differences,  it  is  highly 
unlikely  that  RDA3  and  VAA  would  produce  extremely  similar  acquisition  strategies.  The  models 
differ  significantly  in  their  approach.  They  examine  different  numbers  of  systems,  they  assess 
relative  values  of  systems  differently,  they  draw  cost  data  from  different  sources  and  use  it  at 
different  levels  of  detail  and  they  use  different  mathematical  optimization  formulations  which 
consider  different  objective  functions  and  constraints.  This  constitutes  a  substantial  set  of 
differences  in  their  approach  to  solving  the  RDA  program  trade-off  problem. 

However,  it  is  possible  that  there  could  be  substantial  agreement  between  RDA3  and  VAA.  The 
principal  input  into  the  VAA  objective  function  is  the  system  VAA  coefficient  or  its  measure  of 
contribution  to  force  effectiveness  as  determined  in  the  VAA  Effectiveness  Module.  If  this 
measure  shows  a  high  correlation  with  the  subjective  relative  priority  weights  derived  during 
RDA3’s  Phase  I  Analytic  Hierarchy  Process  (AHP)  survey,  then  substantial  agreement  between 
RDA3  and  VAA  would  be  more  likely.  This  agreement  assumes  that  the  cost  data  used  by  VAA 
and  RDA3,  even  though  drawn  from  separate  sources,  is  fairly  similar.  The  only  method  to  test 
this  possibility  would  be  to  apply  both  models  to  the  same  set  of  systems  and  then  compare  and 
evaluate  the  recommended  acquisition  strategies. 


1.0  Problem  and  Scope  of  the  Study 

1.1  Background  and  Statement  of  the  Problem 

The  Programs,  Analysis  and  Evaluation  Directorate  (PAED)  must  evaluate  a  large  number  of 
research,  development,  and  acquisition  (RDA)  programs  competing  for  limited  Army  resources. 
Traditionally,  program  development  occurs  during  the  Planning,  Programming,  Budgeting,  and 
Execution  System  (PPBES)  Program  Objective  Memorandum  (POM)  building  process  using 
functional  or  mission  area  panels.  These  panels  have  the  latitude  to  establish  their  own  methods 
for  prioritization.  The  lack  of  standardization  prevents  Senior  Army  Leadership  from  making  the 
most  effective  program  trade-offs  across  functional  areas. 

To  provide  an  enhanced  analytical  basis  for  program  trade-off  decisions,  PAED  has  begun  using 
two  models  which  consider  a  wide  range  of  factors  which  affect  the  RDA  cycle  for  all  programs 
under  consideration  for  inclusion  in  the  Army  Program  Objective  Memorandum  (POM).  One 
model,  the  Research,  Development,  and  Acquisition  Alternatives  Analyzer  (RDA3)  was  created  by 
TRADOC  Analysis  Command  -  Operations  Analysis  Center  (TRAC-OAC).  The  other  model. 
Value  Added  Analysis  was  created  by  Concepts  Analysis  Agency  -  Force  Systems  Directorate. 
These  two  models  approach  the  RDA  program  trade-off  problem  in  somewhat  different  ways,  but 
both  can  be  used  to  provide  an  optimized  list  of  RDA  programs  which  should  be  included  in  the 
POM  for  a  given  Total  Obligation  Authority  (TOA). 

Dr.  Ballaschi,  Deputy  Director  of  PAED,  requested  that  the  Operations  Research  Center  (ORCEN) 
compare  the  two  models  to  determine  if  they  would  produce  contradictory  results.  His  concern 
was  that  if  both  models  produce  an  optimized  list  of  RDA  programs  for  inclusion  in  the  POM,  then 
there  should  not  be  significant  disagreement  between  the  output  of  the  two  models. 

1.2  Scope  of  the  Study 

Initial  guidance  provided  to  the  author  was  to  evaluate  the  structure  of  each  model  and  determine  if 
they  should  reasonably  provide  similar  results.  This  study  is  based  on  written  descriptions  of  both 
models  and  conversations  with  personnel  knowledgeable  of  each  model.  The  ORCEN  did  not  run 
each  model  on  a  similar  set  of  inputs  and  subsequendy  evaluate  the  output.  This  study  does  not 
attempt  to  identify  problems  or  evaluate  the  effectiveness  of  either  model  nor  does  it  recommend 
use  of  one  model  or  the  other. 
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2.0  Overview  of  the  Models 

Both  the  Research,  Development  and  Acquisition  Alternatives  Analyzer  and  the  Value  Added 
Analysis  models  address  the  problem  of  recommending  an  optimized  acquisition  strategy.  Yet 
these  models  approach  this  problem  in  different  ways.  In  order  to  compare  and  contrast  the 
approaches  used,  it  is  necessary  to  first  briefly  discuss  the  major  features  of  each  model. 

Additional  detail  for  each  model  may  be  found  in  the  references  cited  at  the  end  of  this  technical 

report. 

2.1  Research,  Development  &  Acquisitions  Alternatives  Analyzer 
(RDA3) 

The  Research,  Development  and  Acquisition  Alternatives  Analyzer  (RDA3)  is  a  combined 
valuation  and  optimization  model  for  helping  the  Army  program  long-term  capital  expenditures. 
RDA3  recommends  optimum  candidate  modernization  investments  by  maximizing  projected  value, 
subject  to  a  variety  of  constraints  on  mission  area  balances,  funding  trajectories,  logical 
relationships  among  programs  and  other  Army  concerns  [Anderson,  et.  al.  1993,  abstract].  RDA3 
is  capable  of  evaluating  all  Management  Decision  Packages  (MDEP)  that  are  considered 
modernization  candidates. 

The  RDA3  is  a  two  phase  decision  support  system.  In  Phase  1  Satty’s  [1980]  Analytic  Hierarchy 
Process  (AHP)  is  used  to  assign  relative  values  to  each  Management  Decision  Package  (MDEP) 
which  is  considered  a  modernization  candidate.  In  Phase  2,  the  relative  MDEP  values  derived  in 
Phase  1  are  combined  with  a  database  that  specifies  the  proposed  annual  MDEP  funding 
increments.  This  information  is  then  input  into  a  multi-objective  goal  program  which  provides  the 
necessary  outputs  to  enable  PAED  action  officers  to  recommend  various  courses  of  action.  Figure 
2.1  presents  a  simplified  overview  of  the  RDA3  decision  support  system  (DSS).  RDA3  will  be 
discussed  in  more  detail  in  subsequent  paragraphs. 

2.1.1  RDA3  -  Phase  1  Process 

Phase  1  begins  with  the  gathering  of  necessary  information.  MDEP  candidates  are  assigned  a 
proposed  funding  structure.  Projected  annual  RD  A  constraints  are  obtained  from  HQD  A.  Other 
considerations,  in  the  form  of  allocation  imperatives  or  goals  are  specified.  The  major  product  of 
Phase  1  however  is  the  determination  of  relative  priority  values  for  each  MDEP  using  the 
hierarchical  assessment  portion  of  RDA3. 

To  assess  the  relative  value  of  one  MDEP  modernization  candidate  in  relation  to  others,  a  number 
factors  must  be  considered.  Issues  such  as  warfighting  contribution,  costs,  field  experience, 
testing  and  evaluation  results,  COEA’s,  OSD  and  Congressional  attitudes,  business  sense,  etc.. 
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must  be  considered  [Anderson,  et.  al.,  1993,  p.8].  This  poses  a  very  difficult  problem.  RDA3 
uses  an  adaptation  of  Satty’s  [1980]  Analytic  Hierarchy  Process  (AHP)  to  obtain  an  Army-wide 
perspective  on  the  potential  value  of  a  modernization  candidate  to  the  future  Army. 
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Figure  2.1  The  RDA3  Process  [Anderson,  et.  al.,  1993,  p.6] 


AHP  is  a  multi-criteria  decision  making  process  which  has  been  used  in  a  wide  variety  of  settings 
to  aid  decision  makers  in  planning,  priority  setting,  and  resource  allocation.  From  a  macro  view 
point,  AHP  has  three  steps.  First  the  problem  is  decomposed  through  the  development  of  a 
hierarchical  structure.  This  decomposition  leads  to  the  establishment  of  a  number  of  criteria  at  each 
level  of  the  hierarchy.  In  the  second  step,  subject  matter  experts  (SME)  are  asked  to  complete  a 
survey.  This  survey  consists  of  a  number  of  judgment  questions  in  which  the  SME  is  asked  to 
judge  the  relative  importance  of  each  criteria  (using  an  ordinal  scale)  through  a  series  of  pairwise 
comparisons.  In  the  final  step,  a  series  of  matrix  calculations  are  performed  using  the  input  from 
the  SME’s  to  derive  estimates  of  the  relative  value  of  candidate  modernization  systems.  A 
thorough  discussion  of  the  AHP  theory  is  contained  in  Saaty  [1980]. 
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The  outcome  of  any  application  of  AHP  is  very  dependent  on  how  the  hierarchical  structure  is 
established.  In  RDA3  a  five  level  hierarchy  based  on  how  Acquisition  Support  and  Program 
Analysis  Directorate  (ASPAD)  of  PAED  evaluates  modernization  systems  is  used  (see  figure  2.2). 
The  top  level  is  the  focus  of  the  overall  effort,  future  Army  modernization.  This  top  level  is 
decomposed  into  the  14  Army  mission  areas  that  ASPAD  uses  when  considering  modernization 
actions.  Each  mission  area  at  the  second  level  is  decomposed  into  sub-mission  areas.  The 
collection  of  about  40  sub-mission  areas  comprises  the  third  level  of  the  hierarchy.  At  the  fourth 
level,  the  modernization  MDEP’s  are  matched  with  the  sub-mission  areas  which  they  support.  The 
bottom  level  of  the  hierarchy  is  formed  by  breaking  out  the  yearly  incrementing  structure  of  the 
individual  MDEP’s. 


Figure  2.2  RDA3  AHP  Hierarchical  Decomposition  Structure  [Anderson,  et.  al.,  1993,  p.9] 

The  outcome  of  the  AHP  process  is  also  affected  by  the  SME  population  selected  to  complete  the 
AHP  surveys.  In  recent  uses  of  RDA3  the  ASPAD  Director  supplied  judgments  for  the  Army 
mission  areas  and  sub-mission  areas.  ASPAD  Action  Officers  supplied  judgments  of  MDEP 
value  relative  to  sub-mission  areas.  A  more  detailed  discussion  of  the  AHP  implementation  in 
RDA3  including  examples  of  SME  questions  and  the  calculation  of  relative  values  is  contained  in 
Anderson,  et.  al.  [1993]. 
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2.1.2  RDA3  -  Phase  2  Process 

In  Phase  2  the  relative  priority  values  determined  through  the  AHP  process  along  with  budgetary 
information  gathered  in  Phase  1  are  placed  into  a  multi-objective  goal  program.  Through  the  use  of 
this  mathematical  programming  technique,  RDA3  attempts  to  structure  a  set  of  MDEP  increments 
that  provides  the  greatest  return  on  investment  for  the  Army,  subject  to  the  investment  dollars 
available  and  other  constraints  which  affect  the  allocation  of  resources  among  competing 
alternatives  [Anderson,  et.  al.,  1993,  p.  163. 

The  goal  programming  model  is  formulated  as  a  combination  of  goals  and  constraints.  The  goals 
represent  target  levels  that  the  math  program  will  attempt  to  achieve,  while  the  constraints  are 
mathematical  conditions  which  must  be  met.  The  RDA3  optimization  module  attempts  to  satisfy 
the  following  goals:  maximize  value  as  derived  from  the  hierarchical  assessment  process;  allocate 
resources  so  as  to  maintain  a  balance  of  resources  among  the  fourteen  Army  mission  areas; 
minimize  turbulence  in  the  year  to  year  funding  profiles  to  prevent  undesired  fluctuating  funding 
oscillations  in  the  MDEP’s  [Anderson,  et.  al.,  1993,  p.16].  There  are  two  types  of  constraints 
used  in  the  optimization  program:  system  constraints  and  logical  constraints. 

System  constraints  relate  to  boundary  conditions  for  the  solution.  Examples  of  system  constraints 
include:  Congressional  /  DoD  MDEP’s  are  funded;  yearly  budgetary  restrictions  are  achieved; 
basecase  MDEP’s  are  funded  before  considering  additional  increments;  recommended  funding  for 
those  MDEP  increments  “selected”  achieves  at  least  a  minimum  threshold  level  of  funding;  upper 
and  lower  bounds  are  maintained  for  the  allocation  of  resources  among  the  mission  areas;  a 
provision  is  made  for  specifying  an  upper  bound  on  operations  and  support  costs  [Anderson,  et. 
al.,  1993,  p.17]. 

Logical  constraints  ensure  required  relationships  between  MDEP’s  are  met.  Examples  of  logical 
constraints  include:  mutually  supportive  MDEP  increments  are  funded  together  or  in  combination 
as  sets;  competing  systems  are  modeled  in  mutually  exclusive  relationships  so  that  no  more  than 
one  system  receives  a  funding  recommendation;  complex  combinations  of  mutually  supportive  and 
competing  MDEP’s  are  selected  in-line  with  decision  maker  wishes  [Anderson,  et.  al.,  1993, 

P- 18]. 

The  formulation  described  above  is  implemented  using  the  General  Algebraic  Modeling  System 
(GAMS)  and  is  run  on  IBM  PC  compatible  386/486  computers.  GAMS  allows  rapid  changes  in 
formulation  of  the  problem.  Solution  times  for  problems  may  be  arrived  at  in  10  -  20  minutes.  The 
rapid  formulation  capability  of  GAMS  makes  it  possible  to  perform  sensitivity  analysis  by  varying 
such  considerations  as,  mandated  MDEP  increments,  logical  constraints,  budgetary  limitations, 
MDEP  valuing,  incremental  funding  thresholds,  and  goals  for  mission  area  funding  levels 
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[Anderson,  et.  al„  1993,  p.21],  A  detailed  description  of  the  optimization  module  is  contained  in 
Donahue  [1992]. 


2.2  Value  Added  Analysis  (VAA) 


The  Value  Added  Analysis  methodology  developed  from  a  perceived  need  to  change ^®^ad^nal 
way  the  Army  has  approached  program  development.  Traditionally,  program  development  has 
been  accomplished  during  the  Planning,  Programming,  Budgeting,  and  Execution ^ 

(PPBES)  POM  building  process  using  functional  or  misston  area  panels.  These  pane  s  often  took 
a  narrow  view  and  optimized  programs  within  their  functional  or  misston  area.  Additionally,  the 
panels  have  had  the  latitude  to  use  their  own  methods  of  prioritization.  This  lack  of  standardization 
has  prevented  Senior  Army  Leadership  from  making  effective  trade-offs  across  functional  areas. 

The  Value  Added  Analysis  (VAA)  methodology  provides  optimized  acquisition  strategies  across 
system  types,  as  well  as  other  analysis  to  support  decision  making  necessary  to  build  the  Army 
POM.  Value  Added  is  defined  as  the  incremental  return  on  investment  as  measured  using 
effectiveness  values  compared  to  cost  [Loerch,  1993].  The  v  aa  metnouoiug, 
interrelated  modules  and  uses  a  family  of  models  to  measure  a  modernization  program 
contribution  to  the  overall  program  as  an  incremental  or  decremental  change  from  the  curren 

program  base. 

The  VAA  process  begins  with  the  Issue  Definition  Module  where  systems  /  programs  to  be  studied 
are  identified  and  issues  such  as  scenarios  and  timeframes  are  resolved.  In  the  Effectiveness 
Module  each  programs  effectiveness  in  the  areas  of  deployability,  sustainment .and  com  a 
effectiveness  is  assessed.  A  detailed  analysis  of  life  cycle  costs  is  performed  for  each  syste 
program  in  the  Cost  Module.  In  the  Optimization  Module  a  cost-benefit  analysis  is  perform  g 
a  mixed  integer  linear  program  optimization  model  to  determine  the  most  cost  effecuve  acqmsttton 
strategy  for  the  candidate  modernization  systems.  The  effectiveness  of  thts  acquisitions  strategy  >s 
modeled  in  a  Theater  Force  Evaluation  Module.  Finally,  in  the  Results  and  Displays  Module 
analysis  of  results  is  conducted,  reports  and  briefings  for  decision  makers  are  preparedlLoerch, 
1993].  Figure  2.3  shows  the  relationships  between  the  VAA  modules. 
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Figure  2.3  Value  Added  Analysis  Modules  [Loerch,  1993] 

The  VAA  methodology  has  been  developed  and  refined  over  the  past  four  years  and  continues  to 
be  extended.  In  particular,  the  Effectiveness  Module  has  been  changed  substantially.  Formerly 
there  were  modules  which  addressed  implicit  effectiveness,  explicit  effectiveness  and  effectiveness 
integration.  In  the  third  VAA  study,  there  will  be  only  one  Effectiveness  Module  in  which  the 
issues  of  system  effectiveness  in  terms  of  deployability,  sustainability  and  combat  effectiveness 
will  be  addressed1.  Additionally,  in  the  coming  year  VAA  will  be  used  to  examine  not  only  RDA, 
but  also  Operations  and  Maintenance,  Army  (OMA)  and  other  related  appropriations  as  needed  to 
investigate  the  full  programming  of  selected  major  item  systems.  Finally,  force  structure 
alternatives  and  levels  of  training  OPTEMPO  will  be  examined  for  possible  inclusion  in  the 


1  Phone  Conversation  by  author  with  LTC  Loerch,  12  Oct  93. 
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analysis.  In  the  end,  70  -  80  systems  should  be  examined  through  the  VAA  process  m  the  next 
POM  build. 

2.2.1  VAA  -  Issue  Definition  Module 

Issue  definition  is  a  process  that  continues  for  the  duration  of  the  Value  Added  Analysis.  Through 
issue  definition,  the  problem  is  refined  so  that  data  collection  and  analysis  efforts  are  focused  on 
the  questions  and  issues  of  interest  to  decision  makers.  During  issue  definition,  decisions  must  be 
reached  concerning  a  number  of  factors  which  will  effect  analysis  conducted  in  subsequent 
modules.  These  factors  include:  The  timeframe  of  the  study;  The  tactical  scenarios  used  to  evaluate 
the  effectiveness  of  systems;  The  major  item  systems  list  -  the  systems  which  will  be  evaluated; 
and  the  Measures  of  Effectiveness  (MOE)  which  will  be  used  in  the  effectiveness  analysis.  See 
Koury  and  Loerch  [1992,  p.3-1  to  3-3]  for  further  details. 

2.2.2  VAA  -  Effectiveness  Module 

Analysis  performed  during  the  Effectiveness  Module  assesses  the  contribution  of  a  system  / 
program  to  overall  force  effectiveness  in  terms  of  three  issues:  deployability;  sustainability;  and 
combat  effectiveness.  Each  of  these  issues  is  evaluated  through  the  use  of  appropriate  models. 

Final  decisions  concerning  which  models  should  be  used  to  evaluate  deployability  and 
sustainability  have  not  yet  been  made.  The  Corps  Battle  Analyzer  (CORB  AN)  is  used  to  evaluate 

combat  effectiveness. 

The  evaluation  of  combat  effectiveness  is  used  to  measure  the  marginal  increase  or  decrease  in 
combat  force  level  performance  a  candidate  system  brings  to  the  total  Army  program.  The  Corps 
Battle  Analyzer  (CORBAN)  is  used  to  determine  combat,  combat  support,  and  combat  service 
support  effectiveness.  The  specific  scenarios  and  MOE's  used  are  determined  in  consultation  with 
decision  makers  during  the  issue  definition  process.  Due  to  limited  computing  resources  and  time 
it  is  not  possible  to  evaluate  all  candidate  systems  in  all  combinations  in  each  year  during  both  the 
POM  period  and  the  extended  planning  period  (EPP)2.  In  previous  uses  of  VAA,  the  evaluation 
time  frame  (which  includes  both  the  POM  and  EPP  periods)  was  divided  into  the  near-term  (1st 
POM  year),  mid-term  (last  POM  year)  and  far-term  Oast  EPP  year).  Thus,  each  selected  scenario 
is  portrayed  in  each  of  the  three  time  frames. 

The  ideal  method  for  determining  the  optimal  mix  of  new  systems  would  be  to  explore  all  possible 
combinations.  To  attempt  to  do  so,  assuming  70  systems  are  to  be  evaluated,  would  require  270  or 
1.18  xlO21  runs  per  scenario-timeframe.  This  is  clearly  infeasible.  VAA  addresses  this  problem 


2The  EPP  is  a  nine  year  period  following  the  last  year  of  the  POM  period. 
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through  the  use  of  a  Plackett-Burman  experimental  design  and  response  surface  methodology  for 
interpreting  outputs.  However,  the  amount  of  time  needed  to  complete  all  runs  for  each  scenario- 
timeframe  is  still  substantial.  See  Koury  and  Loerch  [1992,  p.4-1  to  4-15]  for  additional  detail. 

The  output  of  the  analysis  of  deployability,  sustainability  and  combat  effectiveness  is  a  vector  of 
MOE’s  for  each  system.  Saaty's  [1980]  AHP  is  used  to  construct  a  survey  for  Senior  Army 
Leaders  which  is  then  used  to  determine  the  weighting  scheme  for  the  integration  of  the  selected 
measures  of  effectiveness  and  scenario-timeframe  combinations.  The  weights  obtained  from  the 
Senior  Leader  Survey  are  used  in  an  influence  diagram  whose  output  is  the  value  added 
coefficients.  The  specific  details  of  the  implementation  of  this  process  are  still  being  worked  out 
by  the  VAA  study  team. 

2.2.3  VAA  -  Cost  Module 


The  purpose  of  the  Cost  Module  is  to  provide  accurate  cost  input  to  the  Optimization  Module 
that  will  result  in  reliable  cost  outputs  for  a  wide  range  of  applications.  Weapon  system  cost- 
quantity  relationships,  weapon  system  categorizations,  impacts  on  appropriations  and  formulation 
Life  cycle  costing 


Development 


Production 


Capsule 

(unit  prod,  costs 
or  learning  curve) 


LP 

Optimization 

Model 

Figure  2.4  Overview  of  the  VAA  Costing  Process  [Koury  and  Loerch,  1992,  p.7-1] 
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of  budget  constraints  are  considered  in  the  costing  process  [Koury  and  Loerch,  1992,  p.7- 11- 
Figure  2.4  provides  an  overview  of  the  VAA  costing  process. 

Baseline  Research,  Development,  Test  and  Evaluation  (RDTE)  and  procurement  costs  for  VAA 
systems  are  provided  as  input  to  the  VAA  Optimization  Module.  With  respect  to  procurement 
costs,  systems  are  categorized  as  to  whether  or  not  cost-quantity  curves  (e.g.  learning  curves)  are 
used.  First  unit  costs  and  cost-quantity3  curves  are  provided  for  those  systems  for  which  the  data 
is  available  and  applicable.  Average  unit  procurement  costs  are  used  for  the  remaining  systems. 
Life  cycle  costs  are  based  on  a  baseline  cost  estimate  P-92  cost  code  structure.  Budget  constraints 
are  determined  by  applying  a  ratio,  representing  the  proportion  of  Value  Added  Systems  to  the  total 
RDA  budget  to  a  fixed  TOA  [Koury  and  Loerch,  1992,  p.  7-1  to  7-2]. 

Cost  data  is  obtained  from  a  variety  of  sources  including  the  US  Army  Cost  and  Economic 
Analysis  Center  (CEAC),  Program  Managers  (PM),  the  Assistant  Secretary  of  the  Army  for 
Research,  Development,  and  Acquisition  (ASARDA),  Army  Material  Command  (AMC)  and  the 
Office  of  the  Deputy  Chief  of  Staff  for  Operations  (ODCSOPS).  Most  cost  data  is  extracted  from 
the  Executive  Summary  of  the  Baseline  Cost  Estimate  (BCE)  or  Army  Cost  Position  (ACP). 
Additional  details  on  the  costing  process  may  be  found  in  Koury  and  Loerch  [1992,  Chapter  7  and 
Appendix  E]. 

2.2.4  VAA  -  Optimization  Module 

The  objective  of  the  Optimization  Module  is  to  maximize  the  effectiveness  of  the  force  subject  to 
constraints  on  budget,  force  structure,  and  production  capability.  The  Optimization  Module  uses 
the  Value  Added  Linear  Optimization  of  Resources  (VALOR)  model  as  its  principal  tool  [Koury 
and  Loerch,  1992,  p.8-1].  VALOR  is  a  mixed  integer  linear  program.  The  Effectiveness  Module 
and  Cost  Module  provide  the  necessary  inputs  to  set-up  the  VALOR  model.  Recall,  that  in  the 
Effectiveness  Module  a  single  measure  of  each  system’s  contribution  to  the  effectiveness  of  the 
overall  force  for  each  year  the  system  will  be  in  the  force  is  computed.  This  measure  is  known  as 
the  system’s  value  added  coefficient. 

Since  modeling  in  the  Effectiveness  Module  is  performed  at  discrete  points  in  the  planning  period, 
the  value  added  coefficients  are  only  computed  for  those  years.  VALOR  requires  a  value  added 
coefficient  for  each  year  a  system  could  be  procured.  To  solve  this  problem,  the  assumption  was 
made  that  system  effectiveness  would  vary  linearly  between  the  years  in  which  the  system  is 


3Cost-quantity  reflects  economies  of  scale  in  terms  of  material  and  labor  as  opposed  to  exclusive  learning  on  the  part 
of  the  production  labor  force. 
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actually  modeled.  Linear  interpolation  is  then  used  to  calculate  the  value  added  coefficients  for  the 
years  that  were  not  modeled  in  the  Effectiveness  Module. 

The  objective  function  for  VALOR  is  foimed  as  follows.  Let  vf  be  the  per-item  contribution  of  the 
system  to  force  effectiveness,  and  let  x,  be  defined  as  the  quantity  of  system  i  (of  a  total  of  m 
systems  considered  in  the  study)  procured  in  year  j,  where  jW,  ...,n,  with  n  being  the  num  er  o 
years  in  the  planning  horizon.  The  objective  function  can  then  be  written  as 


n  m 

Maximize 

;= i  i- 1 


This  objective  function  is  subject  to  constraints  which  include:  budgetary  constraints;  Total 
Obligation  Authority;  learning  curve  costs;  nonleaming  curve  variable  costs;  consideranon  of  fixed 
costs;  force  structure  constraints;  production  constraints;  and  related  system l  procuremen 
Additional  details  on  VALOR  may  be  found  in  Koury  and  Loerch  [1992,  Chap.  8]  or  Ae  C 
Technical  Paper  on  VALOR  [CAA  TP  91-7].  VALOR  was  implemented  using  the  ffiM 
Optimization  Software  Library  on  an  IBM  RISC  6000  work  station.  Run  time  for  the  41  systems 
analyzed  in  the  VAA  Phase  II  study  ranged  from  two  to  thirteen  minutes. 


2.2.5  VAA  -  Theater  Force  Evaluation  Module 

The  Theater  Force  Evaluation  Module  is  used  to  evaluate  the  effectiveness  of  a  recommended 
acquisition  strategy.  In  this  module  the  systems  recommended  for  procurement  are  distribute 
throughout  the  total  force.  This  force  is  then  simulated  at  the  Theater  Level  to  evaluate  its 
effectiveness.  This  module  has  not  been  previously  used  in  earlier  VAA  studies. 


3.0  Comparisons  and  Contrasts 

VAA  and  RDA3  were  designed  to  address  the  problem  of  providing  analytical  support  for  system  / 
program  trade-offs  during  the  process  of  budding  the  POM.  However,  due  to  the  different 
approaches  taken  in  approaching  this  problem,  the  models  differ  in  several  ways.  RDA3  and  VAA 
differ  in  their  respective  resolution  and  breadth.  They  have  significant  differences  in  the  methods 
they  use  to  assess  the  relative  value  of  the  systems  /  programs  which  are  being  evaluated.  They 
approach  the  issue  of  obtaining  and  using  cost  data  from  two  very  different  viewpoints.  Finally, 
the  optimization  modules  have  different  objective  functions,  constraints  and  are  usmg  cost  inputs 

derived  from  different  sources. 
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3.1  Model  Resolution  and  Breadth 

RDA3  and  VAA  differ  in  both  the  amount  of  information  or  detail  considered  ( Resolution )  and  in 
the  number  of  systems  which  may  be  evaluated  {Breadth).  In  general,  VAA  might  be  characterized 
as  having  greater  resolution,  but  less  breadth  than  RDA3.  Resolution  differences  between  the  two 
models  are  seen  mainly  in  three  areas:  costing;  optimization;  and  effectiveness  assessment. 

The  cost  data  used  in  the  RDA3  model  is  aggregated  and  rolled  up  into  a  cost  figure  for  each 
MDEP.  The  cost  data  used  in  the  VAA  model  is  much  more  detailed  and  the  optimization  module 
makes  use  of  this  extra  information.  Thus,  VAA  explicitly  considers  additional  details,  such  as 
learning  curve  costs  and  production  -  quantity  cost  relationships  that  are  not  considered  in  the 
RDA3  model.  Additionally,  through  the  use  of  simulation  and  the  resulting  ability  to  perform 
analysis  of  the  results,  VAA  is  able  to  examine  the  effects  of  acquiring  systems  in  greater  detail 
than  the  subjective  AHP  process  used  by  RDA3. 

There  is  also  a  difference  in  the  breadth  of  programs  each  model  can  be  used  to  evaluate.  RDA3  is 
capable  of  evaluating  all  RDA  MDEP’s.  In  recent  uses  RDA3  has  been  used  to  evaluate  329 
MDEP’s4.  VAA  is  constrained  by  its  Effectiveness  Module  and  is  only  capable  of  addressing  a 
subset  of  the  RDA  programs5.  The  specific  systems  to  be  analyzed  in  VAA  are  decided  in 
consultation  with  the  appropriate  decision  makers  during  the  VAA  Issues  Definition  Module.  In 
the  Phase  HI  VAA  study  approximately  70  systems  will  be  evaluated  [Loerch,  1993]. 

3.2  Assessment  of  System/Program  Relative  Value 

The  two  models  have  major  differences  in  how  they  approach  the  assessment  of  relative  value  of 
systems.  RDA3’s  assessment  is  based  on  an  application  of  Saaty’s  [1980]  Analytic  Hierarchy 
Process  applied  to  a  survey  of  Subject  Matter  Experts  from  PA&E.  This  is  necessarily  a  subjective 
approach.  The  VAA  assessment  is  based  primarily  on  analytical  results  obtained  from  simulations 
performed  during  its  Effectiveness  Module.  The  results  of  the  various  analytical  studies  are  then 
combined  using  criteria  value  weights  derived  from  a  survey  of  Senior  Army  Leadership. 

3.3  Costing 

RDA3  and  VAA  approach  the  issue  of  system/program  costing  very  differently.  RDA3  is  MDEP 
focused,  while  VAA  is  system  /  program  focused.  RDA3  obtains  all  costing  data  from  MDEP’s  as 
listed  in  the  Research,  Development,  and  Acquisition  Information  System  Agency  (RDAISA) 


Conversation  with  LTC  Colter,  PA&E,  8  Jul  93. 
5Phone  conversation  with  LTC  Loerch,  CAA,  7  Oct  93 
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database.  The  cost  data  used  is  aggregate  cost  data  and  is  only  broken  out  by  MDEP  increments6. 
VAA  uses  a  great  deal  more  cost  data  and  draws  it  from  a  variety  of  sources.  VAA  draws  most  of 
its  cost  data  from  BCE  and  ACP  estimates  provided  through  CEAC.  Additional  cost  information 
may  be  drawn  from  a  wide  variety  of  agencies,  including  AMC,  ASARDA  and  ODCSOPS.  This 
is  detailed  data  which  covers  production  costs,  learning  curve  costs  and  detailed  life  cycle  costs. 
The  chief  item  to  note  is  that  VAA  costing  is  not  explicitly  based  on  MDEP  costing  and  is  being 
drawn  from  different  sources  than  the  costing  data  used  by  RDA3. 

3.4  Optimization 

The  optimization  modules  in  VAA  and  RDA3  are  also  quite  different.  The  RDA3  optimization 
module  uses  a  multi-objective  weighted  linear  goal  program  which  attempts  to  find  the  best 
acquisition  strategy  which  will  simultaneously  best  satisfy  the  goals  subject  to  the  stated 
constraints.  VAA’s  optimization  module  uses  a  mixed  integer  linear  program  which  is  formulated 
to  maximize  total  force  effectiveness  subject  to  the  stated  constraints.  More  important  perhaps  than 
the  type  of  Linear  Program  (LP)  used  by  each  model  is  the  information  used  to  form  the  objective 
function  and  constraints  in  the  formulation  of  each  LP. 

As  noted  above,  the  cost  data  used  by  VAA  and  RDA3  are  extracted  from  different  sources,  have 
different  levels  of  detail  and  very  likely  do  not  agree  exactly  for  similar  systems.  Note  also,  that 
VAA’s  optimization  module  makes  use  of  cost  data  such  as  learning  curve  cost  and  quantity- 
production  costs  in  deriving  its  optimum  acquisition  strategy  that  are  not  considered  or  modeled 
explicitly  in  RDA3.  Another  major  difference  in  information  inputs  lies  in  the  use  of  relative 
priority  or  effectiveness  values  for  systems.  RDA3  forms  these  values  through  the  AHP  survey 
process.  VAA  forms  relative  values  through  simulation  studies  in  its  Effectiveness  Module. 

A  difference  that  will  significantly  affect  the  recommended  optimum  acquisition  strategy  is  the 
formulation  of  each  model’s  objective  function.  In  RDA3,  the  objective  function  is  formed  as  the 
simultaneous  minimization  of  weighted  and  scaled  deviations  from  the  stated  goals.  Recall  from 
the  discussion  in  Section  2.1.2,  that  the  RDA3  goals  were  to:  maximize  value  as  derived  from  the 
hierarchical  assessment  process;  allocate  resources  so  as  to  maintain  a  balance  of  resources  among 
the  fourteen  Army  mission  areas;  minimize  turbulence  in  the  year  to  year  funding  profiles  to 
prevent  undesired  fluctuating  funding  oscillations  in  the  MDEP’s  [Anderson,  et.  al.,  1993,  p.  16]. 
In  VAA  the  objective  function  is  formulated  to  maximize  total  force  effectiveness7. 


6Yearly  funding  allocations  for  each  budget  or  prgram  year  the  MDEP  covers. 

7See  Section  2.2.4  for  the  mathematical  formulation  of  the  VAA  objective  function. 
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The  last  major  area  that  will  affect  the  output  of  the  optimization  modules  of  VAA  and  RDA3  is  the 
constraints  formulated  in  each  LP.  In  this  area  there  are  a  number  of  similar  constraints^  but  it  is 
important  to  remember  that  the  cost  data  being  used  in  those  constraints  is  derived  from  different 
sources.  However  there  are  also  a  number  of  differences.  RDA3  sets  up  constraints  that  maintain 
balance  across  mission  areas.  This  type  of  constraint  is  not  used  in  VAA.  VAA  uses  constraints 
which  consider  learning  curve  costs,  nonleaming  curve  variable  costs,  consideration  of  fixed 
costs8,  and  quantity-production  costs  which  are  not  used  in  RDA3. 

4.0  Conclusions 

Given  the  differences  between  RDA3  and  VAA  discussed  above  it  is  highly  unlikely  that  they 
would  produce  extremely  similar  acquisition  strategies.  The  models  differ  significantly  in  their 
approach.  They  examine  different  numbers  of  systems,  they  assess  relative  value  of  systems 
differently,  they  draw  cost  data  from  different  sources  and  use  it  at  different  levels  of  detail  and 
they  use  different  mathematical  optimization  formulations  which  consider  different  objective 
functions  and  constraints.  This  constitutes  a  substantial  set  of  differences  in  their  approach  to 
solving  the  RDA  program  trade-off  problem. 

However  it  is  possible  that  there  could  be  substantial  agreement  between  RDA3  and  VAA.  The 
principal  input  into  the  VAA  objective  function  is  the  system  VAA  coefficient  or  its  measure  of 
contribution  to  force  effectiveness  as  determined  in  the  VAA  Effectiveness  Module.  If  this 
measure  shows  a  high  correlation  with  the  subjective  relative  priority  weights  derived  during 
RDA3’s  Phase  1 AHP  survey,  then  substantial  agreement  between  RDA3  and  VAA  would  be  more 
likely.  This  agreement  assumes  that  the  cost  data  used  by  VAA  and  RDA3  even  though  drawn 
from  separate  sources  is  fairly  similar.  The  only  method  to  test  this  possibility  would  be  to  apply 
both  models  to  the  same  set  of  systems  and  then  compare  and  evaluate  the  recommended 
acquisition  strategies. 


8 These  costs  are  not  incurred  on  a  per-unit  basis,  but  rather  on  a  program  basis.  They  represent  RDTE  expenditures 
and  nonrecurring  fixed  manufacturing  costs.  If  a  system  is  not  procured  significant  savings  may  be  accrued  by 
recouping  fixed  costs  funds  that  have  not  yet  been  spent. 
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Appendix  A  -  Glossary 


ACP 

AHP 

AMC 

ASARDA 

ASPAD 

BCE 

CAA 

CEAC 

COEA 

CORBAN 

DoD 

DSS 

EPP 

GAMS 

HQDA 

MDEP 

MOE 

ODCSOPS 

OMA 

OPTEMPO 

ORCEN 

OSD 

PA&E 

PAED 

PM 

POM 

PPBES 

RDA 

RDA3 

RDAISA 

RDTE 

RISC 

SME 

IDA 


Army  Cost  Position 
Analytic  Hierarchy  Process 
Army  Material  Command 

Assistant  Secretary  of  the  Army  for  Research,  Development  and  Acquisition 

Aquisition  Support,  Programs  Analysis  Directorate,  PAED 

Baseline  Cost  Estimate 

Concepts  Analysis  Agency 

US  Army  Cost  and  Economic  Analysis  Center 

Cost  and  Operational  Effectiveness  Analysis 

Corps  Battle  Analyzer 

Department  of  Defense 

Decision  Support  System 

Extended  Planning  Period 

General  Algebraic  Modeling  System 

Headquarters,  Department  of  the  Army 

Management  Decision  Package 

Measure  of  Effectiveness 

Office  of  the  Deputy  Chief  of  Staff  for  Operations 

Operations  and  Maintenance,  Army 

Operational  Tempo 

Operations  Research  Center 

Office  of  the  Secretary  of  Defense 

Programs,  Analysis  and  Evaluation  Directorate 

Programs,  Analysis  and  Evaluation  Directorate 

Program  Manager 

Program  Objective  Memorandum 

Planning,  Programming,  Budgeting  and  Execution  System 

Research,  Development  and  Acquisition 

Research,  Development  and  Acquisition  Alternatives  Analyzer 

Research,  Development  and  Acquisition  Information  Systems  Agency 

Research,  Development,  Test  and  Evaluation 

Reduced  Instruction  Set  Computer 

Subject  Matter  Expert 

Total  Obligation  Authority 
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Appendix  A  -  Glossary 

TRAC-OAC  TRADOC  Analysis  Command-Operations  Analysis  Center 

TRADOC  Training  and  Doctrine  Command 

VAA  Value  Added  Analysis 

VALOR  Value  Added  Liner  Optimization  of  Resources 
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